Papers with information-theoretic approach
Debiasing Methods in Natural Language Understanding Make Bias More Accessible (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent debiasing methods in natural language understanding improve performance on out-of-distribution datasets by pressuring models into making unbiased predictions. |
| Approach: | They propose a general probing-based framework that allows for post-hoc interpretation of biases in language models and use an information-theoretic approach to measure the extractability of certain biase . |
| Outcome: | The proposed framework allows for post-hoc interpretation of biases in language models and measures the extractability of certain biase . |
On the Idiosyncrasies of the Mandarin Chinese Classifier System (N19-1)
Copied to clipboard
| Challenge: | idiosyncrasies of the Chinese classifier system have been studied, but little work has been done to quantify them with statistical methods. |
| Approach: | They propose an information-theoretic approach to measuring idiosyncrasies in Mandarin Chinese by calculating the mutual information between the distribution over classifiers and distributions over other linguistic quantities. |
| Outcome: | The proposed method reduces uncertainty in Mandarin Chinese classifiers by knowing semantic information about nouns that they modify. |
What Do Prosody and Text Convey? Characterizing How Meaningful Information is Distributed Across Multiple Channels (2026.acl-long)
Copied to clipboard
| Challenge: | Prosody—the melody of speech—conveys critical information often not captured by the words or text of a message. |
| Approach: | They propose an information-theoretic approach to quantify how much is conveyed by prosody that is not recoverable from text alone. |
| Outcome: | The proposed framework can quantify how much is conveyed by prosody that is not recoverable from text alone and crucially, what prosody conveys. |